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ABSTRACT 



This exploratory study investigates how writers represent their task to 
themselves before beginning to write. Using data from verbal protocols, we 
examine the initial plans of twelve writers (five experts and seven student writers) 
who were working on an expository writing task. The protocols were coded for 
types of planning. We also obtained independent measures of the quality of the 
subjects' plans and of the quality of their texts. 

The analysis suggests that both the quantity and quality of a vmter's initial 
planning niay make a difference in the quality of the final text. We found a positive 
correlation between the amount of initial planning and text quality, and between the 
quality of plannmg and text quality. In particular, we found that writers who 
developed rhetorical plans (i.e., plans for audience and purpose) tended to produce 
higher-rated texts. From our analysis, we hypothesize that experienced writers 
build a rhetorical representation of their task. We defined a rhetorical representation 
as one which is rich in rhetorical goals and plans relating to the audience, purpose, 
form and language of the text, and in which the writer integrates his plans to form a 
coherent theory of the task. 



L INTRODUCTION 



Writing problems M into the general class of problems characterized by 
Reitman (1964) and Simon (1973) as "ill-structured" problems, analogous to 
problems faced by architects, designers or composers. In contrast to 
"well-structured" problems (e.g., games, or toy tasks) where the goals, operators 
and knowledge required for solving the problem are specified for solvers by the 
task instructions, ill-defined problems require solvers to define their own "problem 
space" as they wo± (Simon, 1973). Dealing with an ill-structured problem, then, 
requires problem-solvers to take an active role in delimiting and defining the 
boundaries of their task, and more specificjdly to: 

• BuUd their own unique representation of their task in response to some 
general (and often vague) task specifications~e.g., "compose a fiigue," 
"write about your job." 

• Specify their own goals and criteria for the task and develop their own 
tests for evaluating possible solutions. 



\^aiile falling within the general category of ill-defined problems, writing 
tasks may vary in the amount of structure imposed by the task instructions. In 
some tasks, like writing a computer manual to tight specifications, some constraints 
(such as format and style) may be given to the writer. In many others, however, 
writers are given a "clean slate" to define their own goals and criteria. In such 
cases, a writer is faced with a myriad of choices: there may be many possible ways 
to represent the task and problem and many possible solutions-i.e., types of text 
that might be produced. The choices that writers make (or do not make) in terms of 
goals, constraints (such as genre) and criteria provide a self-imposed structore to 
the task and guide their search for possible solutions; writers can only accomplish 
the tasks which they set themselves (Floweret al., 1987). 

How then do writers represent their tasks to themselves? What are rhe 
features of their representations and what individual differences might we find? 
Here, we will report the results of an exploratory smdy of the initial task 
representations of twelve writers (with varying amounts of writing expertise) for a 
complex writing task. [This research was part of a major study of experts' and 
novices' planning processes in which we develop a theory of the planning process 
and examine writers' goals and plans before and during composing. See Flower 
et al., 1987; Hayes, 1987]. Before discussing our study of initial task 
representation in depth, let us look at some findings firom other domains which 
provide some working hypotheses about the nature of and differences in 
prr jm-solvers' representations of their task. 

Research in other knowledge-rich domains points to some important 
differences between experts' and novices' problem representations. For a case in 
pomt, let us examine solvers' representations of problems in physics. Although 
physics can be classified as a "fonnal domain" (Larkin, 1983) where the goals and 
operators are more well-defined than in most writing tasks, we can see some 
important analogies with writing: there are several possible problem representations 
that solvers may develop, and the choice of representation greatly affects both the 
efficiency of their search for a solution and the quality of that solution. For 
example, Larkin (1983) and Larkin, McDermott, Simon & Simon (1980) found that 
physics experts build representations of physics problems which are qualitatively 



different from those of physics novices. First, the experts built "elaborated" 
representations which contained a large amount of domain knowledge; second, their 
representations were based on an inter-related set of physics principles which 
Simon and Simon (1978) term "physical intuition." In contrast the novices' 
representations were much more spare and were based on formulae for solving the 
problem rather than on physical principles and concepts. As a result, the novices 
often picked up on cues in the task instructions and jumped straight into the 
problem without conceptualising it; sometimes they were mistaken in their choice of 
formula, and because they did not represent the problem on a more fundamental 
level, they were unable to proceed to a solution. 

This research on physics solvers suggests some interesting questions about 
writers' initial task representations: 

• Do writers* task representations also show qualitative differences 
depending on the writer's expertise and experience? How can we 
characterize these differences? 

• Do expert writers build enriched task representations based on rhetorical 
principles andrelationships-in essence, representations based on what 
we might term "rhetorical intuition"? 

• If so, do such "rhetorical representations" function in a similar way to 
experts' "physical representations"— that is, to allow expert writers to 
structure their task in a way conducive to a successful solution? 

We envisage that a rhetorical representation may provide an expert writer 
with an overall framework for his or her planning process. Thus effective writers 
may develop plans which encompass important rhetorical concepts which they 
instantiate with goals and plans unique to the task at hand. That is, such a rhetorical 
representation would call for the writer to include goals and plans not only for the 
content of the text, but also for its organization, for its overall focus, and for 
meeting the needs of its particular audience and to integrate these components. Thus 
we would expect to see a writer planning a text around these goals and embedding 
content within a purpose. 

This kind of rhetorical representation requires more than a "fill in the blank" 
approach to a writing task- Each new task requires writers to instantiate their 
top-level goals (e.g., to meet the needs of the reader) in a different way. If the 
subject is complex and new, writers may not have available pre-packaged or 
organized pieces of information which can be simply slotted into the text. Rather, 
they will have to engage in difficult knowledge transforming operations to adapt 
what they know to meet fiieir rhetorical goals (Scardamalia & Bereiter, in press). 
We would certainly expect more experienced writers to be more adept at this 
difficult manipulation of content and rhetorical knowledge. For example, in a study 
of young writers, Burtis et al. (1983) found that their subjects' plans consisted 
overwhelmingly of plans for content and that, even when provided with cues for 
more rhetorically oriented planning, the subjects still produced plans for specific 
content or for actual text 

Our analysis has led us to identify five key features of a rhetorical task 
representation. We hypothesize that writers who engage in initial rhetorical 
planning develop plans in the following categories. (The examples presented here 
are from verbal protocols collected from writers working on our experimental 
task-to write about their job for the teenage magazine. Seventeen. We discuss our 
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task, subjects and procedures more fully in the methodology section of this report. 
We also present some extended episodes from our verbal protocols, with an 
analysis of their planning statements, later in the report-see Section m.) 

• Content: What information will the writer include in this text? 

e.g. So maybe I could write about a college suident 
preparing for my major or a job which would be 
public policy and management 

• Form: How will the writer organize and structure this information? 

What genre or format will he or she use? 

e.g. What about a sequence. Reads papers, makes up 
assignments, writes-so forth. I want to start with 
' what they think is obvious 

• Audience: How does the vmter represent the audience (for example, 

their characteristics, atUuides, interests etc.)? 

e.g. Young female teenage audience. They will have 
hadEnglish-audience-they're in school 

» Theme: How does the writer represent the overall focus or rhetorical 
purpose of the text? Does the writer have a unifying idea 
around which other ideas are developed? 

e.g. In feet that might be a useful thing to focus on. 
How a teacher differs from a professor and I see 
myself as a teacher 

• Other goals: What other goals does the writer set for this task? For 
example, what goals does the writer have for addressing the audience, 
for projecting his or her own persona, for the choice of language and 
tone? 

e.g. The tone and style at seventeen. I suppose it should 
be light and lively, filled with slang 



What alternatives to a rhetorical representations might a writer use'> First, 
mexpenenced writers may organize their vmting tasks around content rather than 
rhetorical features; that is, they set themselves the goal to "tell all they know" about 
a topic rather than to adapt or transform their knowledge to meet the rhetorical 
constraints of audience and purpose (Scardamalia & Bereiter, 1982; Scardamalia & 
Bereiter, m press). This "knowledge-telling" strategy may be a useful way for 
young wnters to express their thoughts, but it is clearly inappropriate for complex 

tasks. Tins distinction is similar to that (for adult writers) made by Kem 
et al. (1977) benveen topic-oriented writing and performance-oriented writing; as 
Kern s suidy indicates, where a text will be used to facilitate action or 
decision-making (for example, instructions for operating a machine or policy 
guidelines), topic-oriented writing will not meet the needs of the user/reader 



Second, as we noted earlier, not all writing tasks may require a sophisticated 
rhetorical task representation. For example, take the case of a journalist writing a 
news story. In such a highly structured genre, a standard format--for example, the 
"inverted pyramid" which requires an ordering of information according to 
importances-provides a ready-made set of genre-specific goals, constraints and 
criteria for the task (Schumacher, Klare and Scott, 1985). In such tasks, expert 
writers may not need to build a unique and "customized" representation of the task 
because they already have a well-learned, "off the shelf script which guides and 
constrains their writing process. Their problem-solving efforts, then, may be 
directed to other cognitive activities, such as evaluating the extent to which their 
emerging text meets the genie constraints (Schumacher et aJ., 1985). 

The goals of our study of initial task representation were, then: 

• To investigate possible differences in the type of initial task 
representation our writers built. In particular, we wanted to see if our 
writers built rhetorically-based representations and what kind of 
information they included 

• To investigate whether the type and quality of the initial plans that 
writers make may be related to the quality of their final text; that is, 
whether those writers whose plans had a more rhetorical basis 
produced better texis. 

We wanted to leave open (as far as possible) the goals, constraints and 
criteria for the task to enable writers to build a rhetorical representation if they were 
able, or if they chose, to do so. We thus chose a task which was complex and 
new, and which would then not lend itself to either a knowledge-telling task 
representation or a formula-based representation. 

n. METHODOLOGY 

Subjects 

The subjects for this study were twelve writers with varying degrees of 
expertise and experience. Five were expert writing teachers and seven were student 
writers (four advanced students and three with identified writing difficulties). 

Task 

The subjects were asked to write about their job for Seventeen Magazine. 
They were told that the readers of the magazine would be teenage girls (aged 
thirteen or fourteen). The subjects were given an hour to complete their task, and 
verbal protocols were taken as they worked. 

This task was ill-defined in that each writer could draw on and adapt a 
unique body of personal knowledge about his or her job and develop a unique set 
of goals to reach the young readers of the article. Clearly, there was a variety of 
ways of approaching this task and hence a potential for unique, adaptive solutions 
to a complex rhetorical problem. Our writers on this task had to do what many 
professionals writing to lay audiences have to do: 1) determine their own goals, 
including representing die audience to tiiemselves and deciding how to meet die 
audience's needs; and 2) decide what information to include, given their goals. 
Even the topic of "tiieir job" was open to debate and depended in part on tiieir image 
of the audience and the goals they set for the essay. 



2. These planning clauses were further broken down into the five 

categories which we hypothesized constitute key features or categories 
of planning in a rhetorical task representation: content/topic, form, 
audience, theme, and other goals (see discussion and examples in 
Section 1). Clauses which were simply repetitions of information 
previously stated were excluded from this analysis. 

3. Rating of Planning Quality in the Protocols 

In addition to rating the texts, we obtained a measure of the quality of the 
planning in the initial planning segments of the protocols. Two experienced writers 
rated the protocol excerpts on the three dimensions listed below. The raters were 
explicitly told to consider quality of planning, rather than amount of planning in 
these categories. (See Appendix A for the rating instructions.) 

• How well does the writer's planning reflect a concern for adapting his 
or her text for the audience? 

• To what extent does the writer's planning reflect a concern with 
developing a clear point, focus or rhetorical purpose for his or her text 
that goes beyond simply "knowledge-telling" on a topic? 

• How far does the writer's planning reflect a concern with structuring the 
text or fitting a genre? 

m. RESULTS AND DISCUSSION 

Here we will look at two aspects of our subjects* planning in these initial 
segments of the protocols: 1) the amount of planning that the subjects engaged in 
before beginning to write and 2) the namre and quality of their planning. In our 
analysis, we categorize our subjects in several different ways: 



Category 


Breakdo^vn of Subjects 


Previous writing experience: 


Expert 


Novice 


Amount of initial planning: 

Based on number of planning clauses 

up to first sentence (median split) 


Extensive 
Planners 


Minimal 
Planners 


Quality of final text: 

Based on total quality scores for 

text (median spUt) 


Higher 
Scorers 


Lower 
Scorers 


Type of task representation: 
Based on % of content plans vs. ^ 
rhetorical plans and categories of 
rhetorical plans covered 


Rhetorical 


Content-based 



(We will discuss these categories in more detail as they arise in our analysis.) 



We expected that the above four dimensions would be related-that is, that our 
expert writers would produce texts in the top half of the distribution of text scores 
(high scorers) and that they would develop extensive, rhetorical plans in their 
initial task representations. While two experts did fit this profile, three others did not 
(they were missing one or more dimensions). Similarly, not all our novices were 
"lower scorers"; several of them were extensive planners^ and some did engage in 
rhetorical planning (though perhaps not with the sophistication of our expert 
rhetorical planners). These somewhat suiprising results are discussed further in the 
sections which follow. 

Amount of Initial Planning 

We found a wide range in the amount of planning that the subjects engaged in 
before beginning to write (see Table 1). However, counter to what one might expect, 
the amount of initial planning did not correspond with experience. (There was not a 
significant difference between the number of planning statements for experts and 
novices by the Mann-Whitney test.) The writers who did little planning were not 
always the novices; fo:r example, two of our experts (BS and IB) have only 7 and 2 
planning statements respectj.vely. Rather, we found two modes of planning^ which 
cut across expertise: "extensive," where the subjects do a substantial amount of 
planning and "minimal," where the subjects do little, or almost no planning. 

_ This numerical distinction also appears to reflect two distinct approaches to 
imtial planning which we observed in the protocols. Our six minimal planners 
generally took an approach which can be characterized as "plunge in and write," 
v/hereas our six extensive planners did try to develop some initial plans, with varying 
degrees of success. To illustrate this distinction, we present below examples of the 
initial task representations of two of our typical "minimal" planners-one expert and 
one novice. Neither of these subjects did any initial planning that went bevond 
briefly stating the overall topic for their text, and (in the case of the novice) making 
iDnef references to the audience; both subjects made a decision to b^gin writing early 
m their composing process. Tne expert, in fact, consciously chose to write in order 
to see what (he will) come up with"; and he subsequently only made minor changes 
to the draft that he began at the end of this initial episode. 

Minimal Planner Expert IB (Total planning = 2cls.) 

(Reads and paraphrases assignment). "Okay --let's see..NEH fellow. What's 
NER Okay explain NEH-going to have to do an awful lot of explaining. 
Don t forget the audience. Okay I'm going to try a little prewriting to see what 
I come up with here... NER.um. AIright..charge ahead 
here..charge ahead and see what happens. I'm an NEH fellow at 
CMU- No, job, job, my job. Let's try another tack here" 
(Begins to write a draft) 

Minimal Planner: Novice ED (total planning = 1 i els.) 

(Reads assignment). "I don'i: l.-now where to begin. I don't have any 
thoughts yet. What do they v.'ant to hear...rm talking about these girls.. I 
don t see why I'm going to v/rite different for them than I would (for) 
everybody else. V/hen thiey say my job what do they mean? Presently or 
future, one you want to hold or a job you've had in the past? Maybe a job I 
want to have.. I work in the computer science room at CMU. That's where I 
want to work... Maybe I'll take to the prose and try to write to these 



kids about what they should look for in a job. First of all, the job I want to 
work in is in a business-related environment of the computer science field. If 
possible the department of defense.. Tm going to try to write something that 
they're going to want to read...and just start saying Tm basically looking for 
a job." (Begins to write a draft) 

In contrast, as we can see fix)m Table 1, our extensive planners delayed 
writing a first sentence until they had worked longer on the task. All these writers 
were at least attempting to develop some more substantial initial plans for their text 
(although, there was a large variability in the number of planning clauses excluding 
repetitions in the initial task representations of the "extensive planners": lowest = 21; 
highest = 88). Let us consider the case of novice writer LR (see Table 1) who did the 
least amount of initial plannmg among our extensive planners (21 clauses). In 
comparison with the minimal planners discussed above, writer LR spent more time 
developmg plans for the content of his essay and was thus, we would argue, at least 
attempting initial planning. (In Section IE, we present a transcript of this writer's 
initial task representation and discuss his lack of success in developing many 
rhetorical plans-e.g., plans for audience and purpose-in more depth.) 

Amount of Initial Planning and Text Quality 

We found the amouni of our writers* initial planning to be positively 
correlated with the quality of the text they produced. (See Table 2 for text scores.)^ 
Overall, we found a correlation of .655 between the number of planning statements 
made by our writers and the total quality score for the written texts. For the experts 
the correlation was .694 and for the novices, .799. (These figures were computed 
using the Pearson product moment correlation co-efficient) The high correlation 
between the novices* planning and the quality of their texts is particularly interesting. 
The fact tiiat the two novices who did the least planning (MR with 4 cl. and GJ with 8 
cl.) received the lowest scores for their texts and that two of the novices who did the 
most planning (NK with 44 cl. and SA with 38 cl.) received the highest scores^for 
their texts (out of the novice writers) suggests that the amount of planning before 
beginning to write may be a good predictor of the quality of the text produced when 
an inexperienced writer is facing a relatively complex task. 

We hypothesize that a key variable in predicting success on our msk may have 
been the amount of initial planning rather than prior writing experience.^ We found 
no significant differences (by the Mami-Whitney test) between the quality of the texts 
produced by our expert and our novice writers. Although, the difference between the 
quality of the texts produced by the extensive planners and by the minimal planners 
(see Table 2) was also not statistically significant, though it approached significance 
(p =.0782 by the Mann-Whitney test). However, the positive correlation between 
amount of initial planning and text quality (and especially the high correlation for 
novice writers of .799) does suggest a relationship which warrants further 
investigation. 

Although these results are exploratory, they do suggest that building an initial 
task representation can positively affect the quality of the texts produced. The 
importance of planning throughout a writing task has been observed in previous 
studies (see Flower and Hayes, 1981). This current study suggests that initial 
planning is one aspect of the planning process which contributes to producing a 
successful text, particularly for inexperienced writers. This hypothesis is certainly 
consistent with findings from other domains (such as physics) where novices' 
difficulties often result from alack of time spent conceptualizing the problem before 
beginning to work. The correlation between amount of initial planning and text 
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quality for our experts is more surprising, however. We might expect that 
experienced writers would produce a successful text with little or no initial planning 
because they could draw on well-developed schemas or familiar genres to set then- 
initial goals. However, as we noted earlier, only one of our expert 'writers who did 
"minimal" initial planning received a score in the top-half of the distribution of text 
quality scores (see Table 2; expen BS). One explanation may lie in the ill-defined 
nature of our task which demanded a unique and adaptive set of top-level goals. 

The Quality and Nature of the Initial Task Representations 

, quantity of initial planning alone was too gross a measure of a 

writer's ability to build an effective initial task representation; we were also concerned 
with the quality of the information in these initial planning episodes. Thus, in 
addition to obtaining ratings on the quality of our subjects' final texts, we also 
obtained judgments from different raters on the quality of then- initial plans (up to 
their generation of a first sentence). That is, the raters looked at the quality of the 
planning statements that writers made during the initial segment of the protocol in 
terms of whether the subjects made appropriate plans for addressing the audience, 
whether they developed a clear overall rhetorical purpose and whether they developed 
a clear, cojierent stmcuire. Note that here the raters were not asked to consider the 
amount of planning, but the nature of the rhetorical information in a subjects' plans. 
The ratings for quality of planning' are presented in Table 3. 

We found no significant expen/novice difference in the quality of the 
planning; however, those writers who did more planning (that is, the "extensive" 
planners) obtained significantly higher total quality scores for their planning than did 
the "minimal" planners (p = .0374 by the Mann- Whitney test). Although we were 
measuring quality of planning rather than quantity, it is not very surprising that in 
most cases the extensive planners received higher quality scores than the minimal 
planners simply because they elaborated on their goals more fully; however, it is 
interesting to note that two extensive planners (expert KC and novice LR; see 
Table 3) did receive quality scores at or below the median quality score, which 
supf-sts that although they did some extended planning, then- plans were not very 
rich in rhetorical information. 

An interesting result of this analysis was the very high positive correlation 
between the quality of the planning and the quality of the final texts (.874 by the 
Pearson product moment correlation co-efficient). Thus the writers who were judged 
by one set of raters to have the plans most rich in information were judged 
independently, by a different set of raters, to have the best texts. This result suggests 
that the type and quality' of a writer's planning may well influence the quality of the 
text that he or she produces. The sub-scores for quality of planning for tiie audience, 
purpose and structure showed a positive correlation with the sub-scores for these 
dimensions of text quality as follows: audience = .778; purpose = .741; structure = 
.489. It would seem from these results, then, that concern with audience and purpose 
in planning a text may weU result in texts which are better on these two dimensions. 

Taken along with the strong correlation Letween quantity of planning and text 
quality discussed m the previous section, this relationshio between quality of initial 
plannmg and text quality suggests that, for many writers, the planning which goes 
into building an mmal representation of the task pays off. Although this hypothesis 
requues further testing, we feel that this exploratory analysis at least underscores the 
importance for writers of what we have termed Lnitial "constructive planning." 



ERIC 10 



hfiiimiirnrrTiaaia 



hi 



Types of Initial Planning 



Why does planning seem to help the quality of the texts produced? In looking 
for evidence of the "rhetorical planning" discussed earlier, we found differences in 
the kinds of information embedded in the planning statements. Table 4 shows the 
types of planning information^ generated by higher and lower scorers. (The scores 
are based on qu^ty scores for the text; see Table 2.) Some writers whom we had 
designated as "experts" because they were writing teachers received relatively low 
scores for their texts and were thus placed in the "low scorers" category for this 
analysis; conversely some student writers, originally designated as "novices," 
received scores in the top half of the di^^tribution of scores 9 One explanation for this 
seemingly anomalous result may be that some of the student writers were more easily 
able to relate to the interests of Uieir young readers and thus produced more engaging 
texts than some of the writing teachers. 

First, we looked at whether our writers focused on topic (content) planning or 
on the more rhetorical aspects of planning, such as audience and purpose. We found 
that in terms of absolute numbers our higher scorers simply did more planning in all 
categories, and particularly in the topic category (high scorers' total topic plamiing 
statements = 159; low scorers =- 47), Although the difference in the number of 
plaiuiing statements for high and low scorers was not statistically significant, we did 
firid a positive correlation between the absolute number of topic planning statements 
and text quality (r =.654). This result is not particularly surprising, given the fact that 
four out of the six high-scorers were also extensive planners, and that two subjects in 
this group (WS and BJ) each did twice as much initial planning than any other 
subject 

More surprisingly, when we looked at the proportion of content plans in 
relation to rhetorical plans, we found that, regardless of whether a writer was a 
student or a teacher, the higher scorersJO had a lower percentage (58,67%) of topic 
plans than the lower scorers (67.14%).! 1 fact, we found a negative correlation 
between the percentage of content planning and the total quality score and sub-scores 
for the fmal text (r = -366 for total score; r = -.488 for audience score; r = -,398 for 
purpose score). While these results are by no means conclusive, the inverse 
relationship between the proportion of content planning and text quality may indicate 
that ^vriters may be focusing on planning specfec content, and missing and/or 
ignoring important rhetoric^ goals and constraints, such as adapting a text for an 
audience. For example, the two writers who received the lowest overall scores and 
the lowest scores for audience and purpose (MR and GJ; see Table 2) wrote texts 
which described the technical aspects of their jobs in some detail, but which were not 
geared to the young female teenage audience specified in the assignment 

These differences may suggest some overall differences in the type of initial 
task representations that our writers were building. The fact that the "lower scorers" 
on our task placed greater emphasis on content plans may suggest that at least some 
of them were building a type of "knowledge-telling" or topic-oriented representation 
of their task which ignored the rhetorical constraints of the assignment. In a task that 
called for planning to meet the needs of the audience, some of these writers seem not 
to be developing many rhetorical goals. In contrast, while the "higher scorers" do 
devote about half of their planning to content, they are clearly placing more emphasis 
on developing rhetorical features-or building a "rhetorical representation" of their 
task. 



Second, as Table 4 indicates, we found that all the higher scoring writers 
developed plans in all four rhetorical categories-i.e., theme, form, audience and 
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other goals-as well as topic plans in their initial tdbk representation. In contrast, only 
one of the lower scoring writers covered a]l these categories (and he was a writing 
teacher); the others ignored one or more categories and the two writers who received 
the lowest text scores ignored several categories. The differences between the scores 
of the writers who covered all categories in their initial planning and those who 
ignored one ormore categories was significant by the Mann-Whitney test (p = 
.0149). Furthermore, several of the writers with low quality scores failed to include 
plans for a "theme" or focus for the text We found that those writers who included 
plans in the "theme" category received significantly higher scores for purpose/theme 
on their final texts (p = ,0185 by the Mann-Whitney test). 

Thus we found that the writers who produced the best texts at least "touched 
base" with key rhetorical features, even if like expert BS, they did little initial 
planning. The four least successful novices (i.e., those writers with the lowest scores 
for their texts) demonstrated a lack of concern for rfietorical features, either because 
they did not have the rhetorical knowledge to deal with audience, form etc. or because 
they did not realize that they should do sc. This question of whether some writers fail 
to plan rhetorically because tiiey do not have sufficient knowledge or because they do 
not tiiink to apply then- rhetorical knowledge will be addressed in a further study. 

For a closer look at tiie nature of tiie information in our-writers' initial task 
representation, we will look at some examples in depth. Below are two planning 
episodes fix)m tiie beginning of tiie protocol of one of our more successful expert 
"extensive" planners (WS), whose text was rated highest Altiiough tiiese episodes 
cover orily about a fiftii of tiie task representation he builds before beginning to write, 
tiiey do illustrate some important features of his planning process. We include the 
actual protocol and an analysis of the subject's planning process. 



Writer WS: extensive planner; "higher scorer on text"; expert writing teacher 



Clause # 



Analysis 



Episode 1 



16 Job-English teacherratiiertiian professor 



Topic 



18 In fact that might be a useful thing to 
to focus on 



Theme 
Defines focus 
of text 



20 and I see myself as a teacher 



19 ,..how a teacher differs from a professor 



21 That might help my audience to reconsider 
their notion of what an English teacher 
does 



Goal/theme 
Sets top-level 
goal 



23 
24 
25 
26 
27 



(Reads) young female teenage audience 
They will all have had English 
Audience-tiiey're all in school 
They're taking English 
For many of them English may be a 
favorite subject 



Audience 
Represents 
relevant 
background of 
reader 



29 



But for the wrong reasons-some of then 
may have the wrong reasons in that 
English is good 



Audience 

Represents 

attitudes 



12 



30 Because it's tidy 

31 Can be a neat tidy litde girl 

32 Others turned off because it seems too prim 

33 By God, I can change that notion for them Goal/theme 
Episode 3 



45 All right Im an English teacher 



47 I know they are not going to be 

to be disposed to hear what Tm saying 

48 Partly for that reason and partly to 
put them in the right-the kind of 
frame of mind I want 

49 I v/ant to open with an implied 
question or a direct one 

50 and put them in the middle of some 
situation 

51 then expand from there to talk about 
my job more generally 

52 and try to tie it in with their interest 

53 So one question is where to begin 

54 Start in the middle of-probably the 
first day of class 

55 They'd be interested 

56 They'd probably clue into that easily 

57 because they would identify with the 
first days of school 

58 and my first days are raucous affairs 

59 It would immediately shake 'em up 

60 and get them to think in a different 
Cw.itext 



62 Maybe first 101 class with that crazy 
skit I put on 

63 That's probably better than 305 

64 because 101 is freshmen 

65 and that's nearer to their level 

66 and that sMt was crazy 

67 and it worked beautifully 

68 Okay-let's see before I write that 

69 I think I'll give myself some notion of 
where I'm heading 



Audience 

Represents 
attitudes 

Goal 

(Prepare audience) 
Form 

Develops skeleton 
structure for text 



Goal 

(Involve audienceO 
Form 

Plans introduction 



Audience 

Represents a 
shared reference 

Goal 

Develops specific 
audience goals 
for introduction 

Topic 

Elaborates text 
plan 

Audience 



Process goal 



(Continues to plan up to clause #135 when he ties out a first sentence 

which he rejects. Initial task representation continues 

t( clause #201 when he comes up with a first sentence which he keeps) 
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What features do we see in these excerpts which would lend support to our 
hypothesis that experts build a rhetorical task representation? 

First, this writer's task representation is rich in rhetorical information. 
In these episodes he develops plans and goals in all the categories that we 
hypothesized are key fea$:ures of a rhetorical task representation. Not only does he 
develop plans for specific content to include in his text, but he also develops a theme, 
a partial structure, a quite detailed representation of his audience and a set of 
task-specific goals. For example, he develops a representation of the audience which 
includes their background (they take English in school and may enjoy it) and their 
attitudes (they Bit not going to be disposed to hear what I'm saying). From this 
representation he develops a set of goals for his audience-to "shake them up and get 
them to think in a different context" (clauses 59/60). 

Second, the goals that he generates provide an integrated rhetorical 
framework for his planning. Instead of piecemeal idea generation or brainstorming, 
we see this writer generating and organizing content to meet his particular persuasive 
purpose. For example, he starts out with a scenario which will put the audience in 
the right frame of mind and which will help them to think about English teaching in a 
different context His text plans are thus adapted to fit with his guiding focus. In 
addition, many of his goals interact with each other and two goals may be instantiated 
by a single text plan. For example, he comes up with^the idea to talk about his first 
day of class because this would further his goal to shake up his audience and because 
this would be something his readers could identify with. Thus this one text plan 
instantiates two of his important goals. 

In sum, this writer is generating a set of goals and plans which make this task 
uniquely his own. While the task instructions provide him with some loosely defined 
goals and constraints (the topic, the genre and the audience), he uses the umbrella of 
his rhetorical representation to adapt, elaborate and instantiate these goals. The task 
that he ends up doing is very much a task that he himself has created and a task which 
results from a coherent theory based on rhetorical principles and concepts. 

How does this expert's task representation differ from that of one of the less 
successful writers? Below are some early protocol episodes from one of our less 
successful student writers, whose text was rated eighth out of twelve. 



Writer LR: extensive planner; "lower scorer on text"; novice writer 
Clause # Analysis 
Episode 1 




6 I'm going to assume here I'm an 
engineer 

8 I guess a research-a research engineer 

10 However, there's a graduate student 

1 1 because I'm a graduate student 

12 So ...really ray job is I'm going to school 

13 Let's see-I'm a graduate student 

14 and I'm an engineer 

15 and I'm a research engineer 



14 
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Topic 

Defines subject 
matter of text 

Topic 

(to the end of 
Episode I) 
Explores aspects 
of job 



16 So everything is there 

17 So maybe I should explain here instead 
is that Tm a graduate student pursuing 
a Ph.D. in engineering 

20 I do research work 

22 I teach a course 

Episode 2 

24 (Reads) for a thirteen to fourteen teenage 
audience 

25 So we have to address the fact that this 
girl is seventh or eighth 

27 Okay so these are all the things going 
through my head relative to engineering 
--research engineering 

28 These are all the things Tm supposedly doing 

29 and I'm pursuing a Ph.D, 

30 Working on a thesis 

32 My job here is wrong, the way I'm 
interpreting my job 

34 The way I'm interpreting my job means what 
I'm doing with my life at this moment 
in time 

Episode 3 

37 For the girl is approximately in seventh 
or eighth grade 

38 ••♦the assignment has to appeal to a broad 
range in intellect 

39 It must explain simply what I am doing 

43 I have to generate an essay 

44 We'll assume it's about two pages 

48 That's not really hard 

49 I really have my first line 

50 so I'm going to rip off the page here 
52 I think I can write this out very quickly, 

(Tries out first sentence at clause #53 which he rejects; 
generates first sentence which he keeps at clause #66) 

This novice's task representation looks different from the 
expert's in several ways: 



Audience 



Topic 

Reviews possible 
details to include 



Topic 

Re-defines topic 



Goals 
Sets general 
audience goals 

Form 

Specifies genre 
and length. 



Process 
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First the subject generates few rhetorical goals in contrast to 
the expert's rich rhetorical planning. Rather most of the planning 
is related to features of the topic--e.g., the main aspects of his 
job as a graduate suident in engineering and whether being a 
researcher is in fact his job. He induces no goals for an overall 
theme or focus to the text; little information about organization 
beyond a very general sense of the genre and the length; almost no 
information on his audience's interests or characteristics, beyond 
that provided in the task instructions; and no goals for the text 
beyond a rather vague sense that he should "appeal to a broad range 
in intellect" and that the text "must explain simply what (he is) doing" 
(Clauses 38/39). / . v / 5 

The very general rhetorical goals that he does come up with (e.g., address the 
fact that this girl is in the 7th or 8th grade) are not further instantiated with sub-goals 
and text plans and he is unable to build upon them. 

Second, we do not see the integrated overall framework to guide his planning 
process. His planning seems to be generated haphazardly by features of the 
content with which he is struggling (i.e., a definition of his job), rather than 
mtegrated around rhetorical feanires. Unable to transform his content to meet the 
constraints of the assignment and unable to build his own unique set of rhetorical 
goals for his task, this writer has little option but to "knowledge tell." His text 
reflects this limitation as he produces a rather dry description of energy research 
which our raters judged to be low in meeting the needs of the audience. (See Table. 2 
for text scores.) 

In sum, this writer's planning does not allow him to develop a rhetorical 
focus; eventually, after struggling for some time with his topic knowledge, he simply 
plunges into writing the text, letting his topic information drive his planning and 
generating process. Ahnough he does plan more extensively than many of our other 
subjects, his plans do not help him to carve out a very appropriate and effective 
representation of his task. 

Although we can see limitations in this novice writer's task representation, he 
does produce a text which was rated higher than those of two of our novices whom 
we characterized as "minimal" planners. These writers (MR and GJ; see Tables 1 and 
4) in fact did ahnost no initial planning at all and simply plunged into writing 
immediately. As Table 4 indicates, these subjects' plans were mainly limited to 
content plans, which amounted to a very brief description of their job. Lacking any 
rhetorical goals, these writers had little choice but to knowledge tell, a strategy which 
IS reflected by their low scores for audience and purpose on their written texts. 

IV. SUMMARY AND CONCLUSIONS 

This exploratory analysis of our writers' initial protocol episodes suggests 
that both the quantity and quality of a writer's initial planning may make a 
surpnsmgly large difference in the quality of the final text, in particular for writing 
tasks where the overall goals are only loosely specified (such as our experimental 
task). More specifically, our analysis has resulted in several hypotheses: 
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Writers vary greatly in the amount of planning they do before beginning to write. 
Expert writers do not necessarily do more initial planning than novice writers. 



• There was no significant expert/novice difference in the amount of 
initial planning. 

The quantity of plannmg that a writer does before writing may effect the 
quality of the text he or she produces: the more initial planning, the better 
the final text In general, "minimal** planners are likely to produce worse 
texts than "extensive** planners, p;u:ticularly in the case of inexperienced 
writers. 

• We found a positive correlation between amount of planning and 
text quality: r = .665, all subjects; r = .694, experts; r = .799, 
novices. 

• Although we found no significant difference at the .05 level 
between the total scores; of our extensive and minimal planners 
(p = .0782), we believe that this hypothesis warrants further 
testing with more subjects. 

The quality of a writer*s initial planning (in particular, planning to meet 
the needs of the audience, to develop a purpose and a structure) may well 
affect the quality of the writer*s text. Plans for the audience and purpose 
arc better predictors of success than plans for structure. 

• We found a strong positive correlation between quality- of our 
writers* initial planning and the total quality scores for their texts 
(r = .874). 

• We found a strong positive correlation between: 1) quality of 
audience plans and appropriateness of the text for the audience 
(r = .778); and 2) between quality of plans for purpose and 
purpose score for the text (r = .741). The correlation between 
quality of plans for structure and text score for structure was 
somewhat lower (r = .489). 

Writers who produce successful texts appear to do proportionately less 
**content planning" in their initial task representation, than do those who 
produce less successful texts; conversely, more successful writers* 
mitial plans contain proportionately more rhetorical goals for audience, 
theme, form and for features such as tone and style. However, **better** 
writers may do more content planning in terms of absolute numbers 
because they are more likely to do more planning of all types-i.e., be 
**extensive planners.'* Our results, while by no means conclusive, suggest 
that less successful writers* emphasis on '*content planning*' (as opposed 
to rhetorical planning) results in texts which are less well adapted to the 
audience and which do not manifest a clear rhetorical purpose or 
organizational structure--that is, texts weaker on rhetorical as opposed to 
content features. 
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• We found that our higher scorers did more content planning in 
terras of absolute numbers (159 els. as compared with 47 els.), 
though this difference was not significant. There was a positive 
correlation between number of content plans and text quality 

(r = .654). (This is explained by the fact that four out of six of our 
high scorers were extensive planners, and two did twice as much 
planning (each) than any other subject). 

• We found that those writers whose texts received scores in the top 
half of the distribution of final scores had a lower percentage 
(58.67%) of content planning than did those writers whose scores 
fell in the lower half of the distribution (67.14%). Although these 
figures were not statistically significant (p - .0631), we believe 
that this difference is sufficiently large to warrant fUrther 
investigation. 

• We also found a negative correlation between the percentage of 
content planning and the quality scores for ihe final text: r = -.366, 
total score; r = -.488, audience; r = -.398 purpose. 

Writers who produce more successful texts are .likely to plan in all the 
rhetorical categories discussed earlier when builrding their initial task 
representation, whereas writers with less successful texts are 'ikely 
to ignore some categories. 

• We found that the differences between th(i text scores of the 
writers with plans in all our rhetorical categories and those with 
one or more categories missing was significant (p = .0149). 

• Those writers who included plans for a theme or focus for tlieir 
text received significantly higher scores than those who did not 
(p = .0185). 

Writers who produce more successful texts (in our case, not necessarily 
the exp^ wnting teachers) build a rhetorical representation of their task 
which functions as a coherent theory of the task. Less successful writers 
often do not (or perhaps cannot) build a rhetorical representation of their 
task and may be left with an ineffective knowledge-telling strategy which 
Ignores many of the goals and constraints of the assignment. We defined 
rhetoncal representations to be: 

• rich in rhetorical goals-all the rhetorical categories of planning are 
covered, 

• integrated within a rhetorical fi-amework, and 

• unique for the particular task at hand. 



This exploratory suidy has highlighted the importance of initial planning as a 
possible predictor of text quality. The more successfiil writers simply did more initial 
planning and they did a qualitatively different kind of planning with an emphasis on 
rhetoncal features. We are currently developing suidies to further investigate these 
differences and to address some issues raised by this study. For example, are novice 
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writers unable to develop extensive initial plans and to plan rhetorically because they 
lack the knov^^ledge to do so, or is their apparent failure a matter of executive 
control-i.e., are they failing to plan because they do not prompt themselves 
(or "think") to do so? Certainly research on young writers' cognitive processes 
(Bereiter and Scardamalia, 1982) indicates that failure of executive control is a major 
stumbling block for many developing writers. Our follow-up study presents student 
writers with a series of prompts designed to elicit "expert planning" behavior and 
should provide some further insights into a theory of expert and novice planning 
strategies. 
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FOOTNOTES 

1 We defined the "first sentence" as the first sentence which the writer built upon to 
generate a draft or a partial draft (even if that draft was changed later). Thus 
sentences generated and immediately abandoned were not taken as a "first sentence" 
but as a note. 

2 See Appendix B for a more detailed explanation of this coding scheme. 

^ We should note here that because seven out of our twelve subjects were novices 
we would, of course, expect one novice to obtain a text score above the median and 
to fall mto our extensive planning category. In fact three novices received text scores 
in the top half of the distribution of scores and three were classified as "extensive" 
planners. 

4 We used to median split on the total number of planning clauses (excluding 
repetitions) in the Initial segments of our subjects' protocols (i.e., up to theu- 
generation of a fu^t sentence) to distinguish these two "modes" of planning. 
Extensive planners were those in the top-half of the distribution of the number of 
plannmg clauses; minimal planners were those in the lower half of the distribution 
(See Table 1.) 

5 The scores in Table 2 represent the sums of the scores arrived at by four raters on 
three dimensions: appropriateness for the audience, rhetorical purpose and structure. 
Inter-rater reliability on these scores was .40 on average. Individual figures were 
.619,.243,.116,.375,.536,.451. While these figures are in some cases rather low, 
they seem to reflect a general difficulty in obtaining reliability on overall ratings for 
wntten texts. See Appendix A for a copy of the rating instractions. 

6 As noted earlier, our rating criteria emphasized rhetorical features of the text. We 
suspect that our experts who received relatively low scores for their texts may have 
done so because they were not able to adapt a discussion of their job for the young 
audience m tiie assignment as effectively as some of the student writers. 

7 The scores are the sum of two ratings on three dimensions; audience, purpose and 
organization. Inter-rater reliability on the total scores was .807 by the Pearson 
product moment correlation co-efficient The rating instractions are in Appendix A. 

S Inter-coder reliability on the coding was 71.4%. 

9 We perforaied a median split on the text quality scores to obtain these two groups 
As there were seven novices and five experts we would expect one novice to be 
placed in the higher scorers" group; in fact tiiree received scores in the top half of the 
distnbution of scores. 

10 When we looked at our writers in terms of expert/novice differences, we did find 
that on average the experts (i.e., the writing teachers) also had a lower percentage of 
content planning (51.4%) than did the student writers (73.7%). 

11 These figures were not statistically significant, though the scores for audience 
approached significance by the Mann-Whitney text (p = .0631). 
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Subject 


Total Clauses 


Total Planning 
excluding repetitions 


Planning 
Mode 


Experts 








WS 


201 


88 


Extensive 


BJ 


209 


83 


Extensive 


BS 


24 


7 


Minimal 


KC 


46 


24 




IB 


23 


2 


Minimal 


Novices 








NK 


89 


44 


Extensive 


LR 


67 


21 


Extensive 


SA 


99 


38 


Extensive 


SE 


28 


11 


Minimal 


MR 


o 


A 
*f 




FD 


28 


11 


Minimal 


GJ 


15 


8 


Minimal 



Table 1. Amount of Planning before Beginning to Write 
* Units are clauses. 
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Planning Audience Purpose Structure Total 

Mode 

Experts 



ws 


ext. 


15 


14 


12 


41 


BJ 


ext. 


14 


12 


13 


39 


BS 


min. 


9 


13 


13 


35 


KC 


ext. 


11 


7 


9 


27 


B 


min. 


13 


10 


11 


34 


Novices 












NK 


ext. 


12 


14 


14 


40 


LR 


ext. 


8 


9 


12 


29 


SA 


ext. 


12 


13 


11 


36 


SE 


min. 


15 


12 


9 


36 


MR 


min. 


6 


6 


10 


22 


FD 


min. 


14 


7 


4 


25 


GJ 


min. 


4 


4 


11 


19 



Table 2. Quality Scores for Texts* 



Scores are the sura of four raters who rated the texts on three dimensions 
(audience, puipose and structure). Each dimension was rated on a four point scale 
(l=lowest;4=highest). Highest possible score = 48 ; lowest possible score = 12 
mter-rater rehabiUty on iJiese figures was an average of .4 (highest pair = .619) by 
the Pearson product moment correlation co-efficient 
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Planning Audience Purpose 
Mode 











VYO 


wXU 


0 
0 


Q 
O 






0 
0 


0 




min 




/l 


XVVx 




< 




IB 


rnin 


A 

*T 


L 












wAl* 


Q 
0 


/ 


JLrXV. 


wAl* 


A 




SA 


ext. 


5 


4 


SE 


min. 


8 


5 


MR 


min. 


2 


2 


FD 


min. 


4 


4 


GJ 


min. 


2 


2 



Structure Total Total 

Protocol Text** 

8 24 41 

7 21 39 

5 14 35 

4 13 27 
2 8 34 

7 22 40 

6 14 29 

8 17 36 

5 18 36 

2 6 22 

3 11 25 
2 6 19 



Table 3. Quality Scores for Initial Protocol Excerpts* 



Scores are the sum the scores of two raters who rated the protocols on three dimensions 
(audience, purpose and structure). Each dimension was rated on a four point scale 
(l=lowest; 4=highest). Highest possible total score = 24; lowest possible total = 6. Inter-rater 
reliability on these scores was .807 (on total scores) by Pearson product moment correlation 
coefficient 

The breakdown of these total text scores is presented in Table 2. The correlation 
between total quality scores on the protocols and total quality scores on the text was .874. 



Subject Total* 
High Scorers*** 


Topic 


Theme** 


Form 


Audience 


Other 
Goals 


WS (Expert) 


88 


41 


5 


17 


13 


18 


BJ (Expert) 


83 


46 


3 


25 


3 


9 


BS (Expert) 


7 


3 


2 


1 


2 


1 


NK (Novice) 


44 


35 


3 


4 


3 


2 


SA (Novice) 


38 


29 


1 


3 


1 


4 


SE (Novice) 


11 


5 


1 


2 


1 


2 


TOTAL 


271 


159 


15 


52 


23 


36 


% of total 




58.7% 


5.5% 


19.2% 


8.5% 


13.3% 


Low Scorers 














KC (Expert) 


24 


13 


2 


3 


6 


2 


IB (Expert) 


2 


2 


• 


. 






LR (Novice) 


21 


15 




3 


1 


2 


MR (Novice) 


4 


3 




1 






FD (Novice) 


11 


6 


- 


1 


3 


1 


GJ (Novice) 


8 


8 










TOTAL 


70 


47 


2 


8 


10 


5 


% of total 




67.1% 


2.8% 


11.4% 


14.3% 


7.1% 



Table 4. Initial Task Representation: Types of Planning 



Total planning clauses excluding repetitions. 

Clauses coded as theme were also coded in another category so percentages total 
more than 100%. 



Subjects with scores in the top half of the distribution. 



/CO 
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APPENDIX A 



Rating Instructions for Texts 

Hie attached texts were produced by twelve writers who were given the 
following assignment: 

Describe your job for a young teenage female audience (ages 13-14). 
Your text will appear in Seventeen Magazine. 

As you will see, some of the texts were written by students and some by 
teachers. We are interested in your judgments about the quality of the texts, 
independent of whether the writer is a student or a teacher. 

Please rate the quality of the texts according to three criteria: audience, 
purpose and structure. (A more detailed explanation of these criteria is given 
below.) Each text should receive three scores, each on a four-point seals. 

Please consider each of the three ratings on separate passes-that is, rate all 
the texts for audience, then all the texts for purpose, and then all the texts for 
structure. Some texts may be good on one dimension, but not on others. 

In many cases raters tend to be conservative in their judgments and use only 
the middle points of tlie scale. Here we would like you to spread your judgments 
oyer the whole range of the scale. (We would really like to make some clear 
distinctions between the relative quality of these texts on the three rating 
dimensions.) 

Rating Criteria 



1. How well is the text adapted for the audience specified in the assignment? 

1 2 3 4 

Not adapted Very weU adapted 

at all for audience for audience 

2. To what extent does this text have a clear point, focus or rhetorical 
purpose that goes beyond simply "knowledge-telling" on a topic? 

1 2 3 4 

No clear point/ Very clear point/ 

focus/puipose focus/puipose 

3. How well-structured is the text? Consider both overall organization and 
coherence. 

1 2 3 4 

, Very well-structured 

well-structured 
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APPENDIX B 



Rating Instructions for Protocol Excerpts 



The attached protocol excerpts contain the initial plans (i.e., plans generated 
before writing a first sentence*) of twelve writers who were given the following 
assignment: 

Describe your job for a young female teenage audience (ages 13-14). 
Your text will appear in Seventeen Magazine. 

As you will see, some of the writers were students and some were teachers. In 
addition, some of our subjects did most of their planning before beginning to write, 
whereas others did most of their planning during the process of generating a draft. Here 
we are interested in your judgments about the quality of the planning in these initial 
protocol excerpts, independent of whether the writer was a student or a teacher. 

Please rate the quality of the writers' plaiuiing in these protocol excerpts 
according to three criteria: audience, purpose and structure. (These criteria are explained 
in more detail below.) Each excerpt should receive three scores, each on a four-point 
scale. 

Please consider each of the three ratings on separate passes that is, rate all the 
excerpts for audience, then all the excerpts for purpose and then all the excerpts for 
structure. Some of the excerpts may be good on one dimension, but not on others. 

In many cases raters tend to be conservative in their judgments and to use only the 
middle points on the scale. Here we would like you to spread your judgments over the 
whole range of the scale. (We would really like to make some clear distinctions between 
the relative quality of the writers' planning on the three rating dimensions.) 

Rating Criteria 

1 . How well does the writer's planning reflect a concern for 
adapting his/her text for the audience specified in the assignment? 



2. To what extent does the writer's planning reflect a concern with developing a 
clear point, focus or rhetorical purpose for his/her text that goes beyond 
simply "knowledge-telling'' on a topic? 



1 



2 



3 



No concern 
with adapting 
to the audience 



Very concerned with 
adapting to the 
audience 



1 

No concern 
with point/ 
focus/purpose 



3 



4 

Very concerned with 
point/focus/purpose 
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3. How far does the writer's planning reflect a concern with structuring the text 
orfittingagenre? 



1 2 3 4 

No concern Very concerned with 

with structure structure 



Note: These excerpts end at the point where the subjects wrote their first sentence of 
text (For the purpose of this analysis, the "first sentence" was defined as the first 
sentence which the writer built upon to generate a first draft. Sentences which were 
generated and then immediately rejected were considered "notes" rather than text.) 
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